Papers with cross-language-family audio-lip alignment
AudioVSR: Enhancing Video Speech Recognition with Audio Data (2024.emnlp-main)
Copied to clipboard
Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin
| Challenge: | Recent work has shown poor performance with non-Indo-European languages . previous work primarily utilizes video information to build VSR models . |
| Approach: | They propose a generative model for data inflation that integrates synthetic data with authentic visual data to enhance the VSR model. |
| Outcome: | The proposed model improves on the audio-visual alignment problem in audio-video tasks. |